Generating column offset corrections for image sensors

ABSTRACT

An image sensor includes multiple photoactive pixels and multiple dark reference pixels typically arranged in rows and columns to form a pixel array. A dark signal is read out from a given number of dark reference pixels in each column at a first gain level. An initial column offset correction is determined for one or more columns in the pixel array using respective dark signals read out at the first gain level. The initial column offset corrections are repeatedly scaled in response to each detected change to a different gain level. The column offset corrections can be scaled based on an amount of change between each respective different gain level and the first gain level.

TECHNICAL FIELD

The present invention relates generally to image sensors, and moreparticularly to methods for compensating for column fixed pattern noisein image sensors. Still more particularly, the present invention relatesto methods for generating column offset corrections used to compensatefor column fixed pattern noise in image sensors.

BACKGROUND

Complementary Metal Oxide Semiconductor (CMOS) image sensors typicallyinclude multiple pixels that are arranged in rows and columns to form apixel array. A column output circuit is connected to each column ofpixels in the array to read out the signals from the pixels in thecolumn. The output signals from a row of pixels are read out of thepixel array one row at a time (in parallel) by the column outputcircuits.

The column output circuits can include both analog and digital circuits,including capacitors, switches, multiplexers, transistors, andamplifiers. The quality of an image captured by a CMOS image sensor canbe reduced due to differences or mismatches between the offsets and thegains of the circuitry in the column output circuits. These mismatchesproduced column fixed pattern noise artifacts or defects in the images.Column fixed pattern noise is more visible and objectionable to anobserver due to the columnar or striped structure of the column fixedpattern noise.

When an image is captured, a gain is often applied to the output signalsof the pixels. This is especially true when the image is captured in lowlight conditions, where the gain is used to make the image brighter.Unfortunately, column fixed pattern noise is also amplified by the gain,making the column fixed pattern noise artifacts even more noticeable inan image. For example, a twofold increase in the gain can result in adoubling of the column fixed pattern noise.

SUMMARY

The present invention relates to a method for determining column offsetcorrections used to compensate for column fixed pattern noise in imagescaptured by an image sensor. The image sensor includes multiplephotoactive pixels and multiple dark reference pixels typically arrangedin rows and columns to form a pixel array. A dark signal is read outfrom a given number of dark reference pixels in each column at a firstgain level. An initial column offset correction is determined for one ormore columns in the pixel array using respective dark signals read outat the first gain level. The initial column offset corrections arerepeatedly scaled in response to each detected change to a differentgain level. The column offset corrections are scaled based on an amountof change between each respective different gain level and the firstgain level in an embodiment in accordance with the invention.

BRIEF DESCRIPTION OF THE DRAWINGS

Embodiments of the invention are better understood with reference to thefollowing drawings. The elements of the drawings are not necessarily toscale relative to each other.

FIG. 1 is a simplified block diagram of an image capture device in anembodiment in accordance with the invention;

FIG. 2 is a block diagram of a top view of image sensor 106 in anembodiment in accordance with the invention;

FIG. 3 is a block diagram of analog front end circuit 216 shown in FIG.2;

FIG. 4 is a flowchart of a first method for generating column offsetcorrections in an embodiment in accordance with the invention;

FIG. 5 is a flowchart of a first method for using the column offsetcorrections generated according to the first method shown in FIG. 4;

FIGS. 6A-6B depict a flowchart of a second method for generating andusing column offset corrections in an embodiment in accordance with theinvention;

FIGS. 7A-7B illustrate a flowchart of a method for detecting outlierdark signals and compensating for the outlier dark signals whengenerating column offset corrections in an embodiment in accordance withthe invention; and

FIG. 8 is an exemplary plot of dark signals read out of each column thatis used to depict local offset windows in an embodiment in accordancewith the invention.

DETAILED DESCRIPTION

Throughout the specification and claims, the following terms take themeanings explicitly associated herein, unless the context clearlydictates otherwise. The meaning of “a,” “an,” and “the” includes pluralreference, the meaning of “in” includes “in” and “on.” The term“connected” means either a direct electrical connection between theitems connected, or an indirect connection through one or more passiveor active intermediary devices. The term “circuit” means either a singlecomponent or a multiplicity of components, either active or passive,that are connected together to provide a desired function. The term“signal” means at least one current, voltage, or data signal.

Additionally, directional terms such as “on”, “over”, “top”, “bottom”,are used with reference to the orientation of the Figure(s) beingdescribed. Because components of embodiments of the present inventioncan be positioned in a number of different orientations, the directionalterminology is used for purposes of illustration only and is in no waylimiting.

Referring to the drawings, like numbers indicate like parts throughoutthe views.

FIG. 1 is a simplified block diagram of an image capture device in anembodiment in accordance with the invention. Image capture device 100 isimplemented as a digital camera in FIG. 1. Those skilled in the art willrecognize that a digital camera is only one example of an image capturedevice that can utilize an image sensor incorporating the presentinvention. Other types of image capture devices, such as, for example,cell phone cameras, scanners, and digital video camcorders, can be usedwith the present invention.

In digital camera 100, light 102 from a subject scene is input to animaging stage 104. Imaging stage 104 can include conventional elementssuch as a lens, a neutral density filter, an iris and a shutter. Light102 is focused by imaging stage 104 to form an image on image sensor106. Image sensor 106 captures one or more images by converting theincident light into electrical signals. Digital camera 100 furtherincludes processor 108, memory 110, display 112, and one or moreadditional input/output (I/O) elements 114. Although shown as separateelements in the embodiment of FIG. 1, imaging stage 104 may beintegrated with image sensor 106, and possibly one or more additionalelements of digital camera 100, to form a camera module. For example, aprocessor or a memory may be integrated with image sensor 106 in acamera module in embodiments in accordance with the invention.

Processor 108 may be implemented, for example, as a microprocessor, acentral processing unit (CPU), an application-specific integratedcircuit (ASIC), a digital signal processor (DSP), or other processingdevice, or combinations of multiple such devices. Various elements ofimaging stage 104 and image sensor 106 may be controlled by timingsignals or other signals supplied from processor 108.

Memory 110 may be configured as any type of memory, such as, forexample, random access memory (RAM), read-only memory (ROM), Flashmemory, disk-based memory, removable memory, or other types of storageelements, in any combination. A given image captured by image sensor 106may be stored by processor 108 in memory 110 and presented on display112. Display 112 is typically an active matrix color liquid crystaldisplay (LCD), although other types of displays may be used. Theadditional I/O elements 114 may include, for example, various on-screencontrols, buttons or other user interfaces, network interfaces, ormemory card interfaces.

It is to be appreciated that the digital camera shown in FIG. 1 maycomprise additional or alternative elements of a type known to thoseskilled in the art. Elements not specifically shown or described hereinmay be selected from those known in the art. As noted previously, thepresent invention may be implemented in a wide variety of image capturedevices. Also, certain aspects of the embodiments described herein maybe implemented at least in part in the form of software executed by oneor more processing elements of an image capture device. Such softwarecan be implemented in a straightforward manner given the teachingsprovided herein, as will be appreciated by those skilled in the art.

Referring now to FIG. 2, there is shown a block diagram of a top view ofimage sensor 106 in an embodiment in accordance with the invention.Image sensor 106 includes a pixel array 200 that has photoactive pixels202 and dark reference pixels 204. The photoactive and dark referencepixels 202, 204 are typically arranged in rows and columns within pixelarray 200.

Photoactive pixels 202 are pixels that collect photo-generated chargecarriers in response to incident light. Dark reference pixels 204 may beconstructed as photoactive pixels with an opaque layer or light shield206 positioned over dark reference pixels 204 so they do not receiveincident light. Alternatively, dark reference pixels 204 can beconstructed without photodetectors in an embodiment in accordance withthe invention.

Dark reference pixels 204 are used to measure the amount of chargeproduced in image sensor 106 when the image sensor is not illuminated(zero light conditions). These dark signals are employed to reduce theamount of noise in captured images. In embodiments of the presentinvention, column offset corrections for each column in pixel array 200are generated based on the dark signals read out of dark referencepixels 204. The column offset corrections are used to reduce oreliminated column fixed pattern noise in image sensors.

Image sensor 106 further includes column decoder 208, row decoder 210,digital logic 212, column output circuits 214, and analog front endcircuit 216. Each column of photoactive and dark reference pixels 202,204 in pixel array 200 is electrically connected to a column outputcircuit 214.

Digital logic 212 includes memory 218, control register 220, scalingcircuitry 222, and timing generator 224. In an embodiment in accordancewith the invention, memory 218 stores the column offset corrections,control register 220 stores the gain level used when an image iscaptured, and scaling circuitry 222 is used to scale the column offsetcorrections in response to a detected change in a gain level. Timinggenerator 224 generates the signals needed to read out signals frompixel array 200.

Image sensor 106 is implemented as an x-y addressable image sensor, suchas, for example, a Complementary Metal Oxide Semiconductor (CMOS) imagesensor, in an embodiment in accordance with the invention. Thus, columndecoder 208, row decoder 210, digital logic 212, column output circuits214, and analog front end circuit 216 are implemented as standard CMOSelectronic circuits that are operatively connected to pixel array 200.

Functionality associated with the sampling and readout of pixel array200 and the processing of corresponding image data may be implemented atleast in part in the form of software that is stored in memory 110 (seeFIG. 1) and executed by processor 108. Portions of the sampling andreadout circuitry may be arranged external to image sensor 106, orformed integrally with pixel array 200, for example, on a commonintegrated circuit with photodetectors and other elements of the pixelarray. Those skilled in the art will recognize that other peripheralcircuitry configurations or architectures can be implemented in otherembodiments in accordance with the invention.

For simplicity, FIG. 2 depicts seven rows and seven columns of pixels,with five rows of photoactive pixels 202 and two rows of dark referencepixels 204. Those skilled in the art will recognize image sensors havemillions to tens of millions of pixels that can be arranged in anyconfiguration. By way of example only, rows of dark reference pixels canbe situated at the top and bottom of pixel array 200. Alternatively, thephotoactive pixels can be confined in a sub-array with rows and columnsof dark reference pixels surrounding the sub-array. Another exemplaryembodiment disperses the dark reference pixels within the pixel arraysuch that dark reference pixels are intermingled with photoactivepixels.

FIG. 3 is a block diagram of analog front end circuit 216 shown in FIG.2. Analog front end circuit 216 receives a differential pair of analogsignals from each pixel in an embodiment in accordance with theinvention. One analog signal is identified as RESET and the other signalas SIGNAL. Analog front end circuit 216 amplifies and conditions theRESET and SIGNAL analog signals, and converts the analog signals todigital signals.

Analog front end circuit 216 includes analog to digital converter (ADC)300 and analog signal processor (ASP) 302. ASP 302 includes two cascadedvariable gain amplifiers 304, 306 in an embodiment in accordance withthe invention. Other embodiments in accordance with the invention caninclude one or more variable gain amplifiers. Changes in the gains ofvariable gain amplifiers 304, 306 are detected in the present inventionand used to produce scaled column offset corrections in an embodiment inaccordance with the invention.

Referring now to FIG. 4, there is shown a flowchart of a first methodfor generating column offset corrections in an embodiment in accordancewith the invention. Initially, dark signals from a given number of darkreference pixels in each column of pixels are read out at a given gainlevel (block 400). The gain level can be a gain setting or a measuredgain in embodiments in accordance with the invention.

The number of dark reference pixels read out can vary in embodiments inaccordance with the invention. In one embodiment, dark signals are readout of a large number of dark reference pixels in each column. A highnumber of dark signals can result in more precise initial column offsetcorrections.

Moreover, the given gain level can be any desired gain level. The gainlevel is high in one embodiment in accordance with the invention. Ahigher gain level can produce more accurate initial column offsetcorrections.

Block 400 can be performed at any time during the operation of an imagesensor. Block 400 is first performed when an image sensor is turned onin one embodiment in accordance with the invention. This allows theblanked frame that is part of the start up sequence to be used forobtaining the dark signals.

Next, as shown in block 402, the dark signals from each column areanalyzed to determine if there are any outlier dark signals. An outlierdark signal is a signal obtained from an unusually dark or light darkreference pixel in an embodiment in accordance with the invention. Byway of example only, digital logic 212 (FIG. 2) analyzes the darksignals to determine if any are outlier dark signals. A method fordetecting and compensating for outlier dark signals is described laterwith reference to FIGS. 7A-7B. If any of the dark signals are outlierdark signals, the outlier dark signals are discarded or compensated forin embodiments in accordance with the invention.

An initial column offset correction is then determined for each columnin the pixel array and stored in a memory (blocks 404 and 406). The darksignals read out of a column are averaged together to produce an averagedark signal value, and the average dark signal value is used as aninitial column offset correction in one embodiment in accordance withthe invention. Other embodiments in accordance with the invention candetermine the initial offset corrections differently. By way of exampleonly, the dark signals can be input into an infinite impulse response(IIR) filter to determine the initial column offset corrections.

A determination is then made at block 408 as to whether or not imagecapture has been initiated. The gain level is determined by an imagecapture device when the image capture process is initiated in anembodiment in accordance with the invention. The gain level can bedetermined by an automatic exposure algorithm, by user selection, or bysome other method. The gain level is transmitted to the image sensor andstored in control register 220 (FIG. 2) in an embodiment in accordancewith the invention.

A determination is then made at block 410 as to whether or not the imagewill be captured with a different gain level than the gain level used atblock 400 and block 404. Digital logic 212 detects changes in the gainlevel in an embodiment in accordance with the invention. If the gainlevel has not changed, the method returns to block 408. If the imagewill be captured with a different gain level, the process continues atblock 412 where the initial column offset corrections are scaled inresponse to the different gain level. The initial column offsetcorrections are scaled by the ratio of a new gain setting to a previousgain setting in one embodiment in accordance with the invention. Inanother embodiment in accordance with the invention, the initial columnoffset corrections are scaled by the ratio of a current measured gain toa previous measured gain.

The scaled offset corrections are then stored in memory, as shown inblock 414. The method returns to block 408. Blocks 412 and 414 repeateach time an image is captured with a different gain level in anembodiment in accordance with the invention.

The method of FIG. 4 updates the column offset corrections when a changein a gain level is detected. The column offset corrections are updatedby scaling the initial column offset corrections. This methodadvantageously avoids repeatedly reading out dark signals whendetermining column offset corrections, thereby reducing the frameoverhead (e.g., time) needed to compute a column offset correction foreach column in the pixel array.

Embodiments in accordance with the invention are not limited to theblocks and the order of the blocks shown in FIG. 4. Other embodiments inaccordance with the invention can perform additional steps, not performsome of the blocks, or perform some of the steps concurrently. By way ofexample only, detecting outlier dark signals and compensating for suchoutlier dark signals (block 402) can be executed concurrently withreading out dark signals (block 400). Alternatively, block 402 or block414 does not have to be performed in other embodiments in accordancewith the invention.

FIG. 5 is a flowchart of a method for using the column offsetcorrections generated according to the method shown in FIG. 4.Initially, a determination is made as to whether or not an image orframe of an image is to be read out of a pixel array (block 500). If so,the appropriate column offset correction is subtracted from the imagesignals read out of each column (block 502). The appropriate columnoffset correction is the initial column offset correction or apreviously scaled initial column offset when the image or the frame ofan image is captured without a change in the gain level. Alternatively,the appropriate column offset correction is a currently scaled columnoffset correction when the image or frame of an image is captured with adifferent gain level than the gain level associated with the initialcolumn offset correction.

The scaling of the initial column offset corrections in response to achange in gain level (block 412 in FIG. 4) can be done concurrently withsubtracting the appropriate offset correction from image signals (block502 in FIG. 5) in one embodiment in accordance with the invention. Thatis, the appropriate offset correction can be determined and scaled (ifnecessary due to a change in gain) on-the-fly as needed during readoutof a frame or image in order to provide offset correction to each imagesignal as it is read out. This avoids the need for a memory to store thescaled offset correction.

Referring now to FIGS. 6A-6B, there is shown a flowchart of a secondmethod for generating and using column offset corrections in anembodiment in accordance with the invention. Initially, dark signalsfrom a given number of dark reference pixels in each column of pixelsare read out at a given gain level (block 600). The gain level can be again setting or a measured gain in embodiments in accordance with theinvention.

The number of dark reference pixels read out can vary in embodiments inaccordance with the invention. Dark signals are read out of a few darkreference pixels in each column in one or more embodiments in accordancewith the invention. For example, in one embodiment in accordance withthe invention, dark signals are read out of thirty-six dark referencepixels in each column. In another embodiment in accordance with theinvention, dark signals are read out of eight dark reference pixels ineach column.

The given gain level can be any desired gain level. The gain level ishigh in one embodiment in accordance with the invention. A higher gainlevel can result in more accurate initial column offset calculations.

Block 600 can be performed at any time during the operation of an imagesensor. Block 600 is first performed when an image sensor is turned onin one embodiment in accordance with the invention. This allows theblanked frame that is part of the start up sequence to be used forobtaining the dark signals.

Next, as shown in block 602, the dark signals from each column areanalyzed to determine if there are any outlier dark signals. Asdiscussed earlier, an outlier dark signal is a signal obtained from anunusually dark or light dark reference pixel in an embodiment inaccordance with the invention. A method for detecting and compensatingfor outlier dark signals is described later with reference to FIGS.7A-7B. If any of the dark signals are outlier dark signals, compensationdark signals are used or the outlier dark signals are discarded inembodiments in accordance with the invention.

A column offset correction is then determined for each column in a pixelarray, as shown in block 604. In one embodiment in accordance with theinvention, the dark signals read out of each column are averagedtogether to produce a column average dark signal value, and the columnaverage dark signal value is used as a column offset correction. Thecolumn offset corrections are stored in a memory (block 606).

A determination is then made as to whether or not image capture has beeninitiated (block 608). As previously described, the gain level isdetermined by an image capture device when the image capture process isinitiated in an embodiment in accordance with the invention. The gainlevel can be determined by an automatic exposure algorithm, by userselection, or by some other method. The gain level is transmitted to theimage sensor and stored in control register 220 (FIG. 2) in anembodiment in accordance with the invention.

A determination is then made at block 610 as to whether or not the imageis captured with a different gain level than the gain level associatedwith the column offset corrections stored in memory at block 606 or atblock 618, whichever occurred last. Digital logic 212 detects changes inthe gain level in an embodiment in accordance with the invention.

When the image is not captured with a different gain level, the processcontinues at block 612 where a given number of dark signals are read outof the dark reference pixels. The given number of dark reference pixelsread out of the pixel array can change dynamically each time darksignals are read out of dark reference pixels in embodiments inaccordance with the invention. Thus, one read operation may read outeight dark reference pixels to determine a column offset correction andanother read operation may read out ten dark reference pixels. By way ofexamples only, the number of dark signals read out of the pixel arraycan be based on noise in the dark reference pixels, on statisticalmeasurements on the column offset corrections, on whether the pixels arebeing read for an update after the gain level changes for the firsttime, or whether the dark reference pixels are being read for asubsequent update.

Next, after the given number of dark pixels are read, the dark signalsfrom each column are analyzed to determine if there are any outlier darksignals (block 614). A method for compensating for outlier dark signalsis described later with reference to FIGS. 7A-7B. The previous columnoffsets are then updated to produce updated column offsets, as shown inblock 616. The updated column offset corrections are stored in memory(block 618).

In one embodiment in accordance with the invention, the column offsetcorrections are re-calculated using the newly read out dark signals andthe previously computed column offset corrections. By way of exampleonly, the average dark signal level for each column is re-calculatedeach time the given number of dark signals are read out of the pixelarray. The re-computed column offset corrections are used as updatedcolumn offset corrections. Other embodiments in accordance with theinvention can produce updated column offset corrections differently. Byway of example only, the updated column offsets can be produced using aweighted average or an IIR filter in other embodiments in accordancewith the invention.

Each updated column offset correction is then subtracted from the imagesignals read out of the column associated with the updated column offsetcorrection (block 620). Subtracting the updated column offsetcorrections from the image signals compensates for column fixed patternnoise. The method then returns to block 608.

Referring again to block 610, the process passes to block 624 when adifferent gain level is used when capturing an image. At block 624 thecolumn offset corrections or the updated column offset corrections arescaled to produce scaled column offset corrections. The amount ofscaling is based on a detected change in the gain level. For example,the column offset corrections or updated column offset corrections arescaled by the ratio of a new gain setting to a previous gain setting inone embodiment in accordance with the invention. In another embodimentin accordance with the invention, the column offset corrections or theupdated column offset corrections are scaled by the ratio of a currentmeasured gain to a previous measured gain.

The scaled column offset corrections are then stored in memory (block626). After block 626 is executed, the process passes to block 612 andblocks 612 through 620 are performed. After block 620 is executed, themethod returns to block 608.

Embodiments in accordance with the invention are not limited to theblocks and the order of the blocks shown in FIGS. 6A-6B. Otherembodiments in accordance with the invention can perform additionalsteps, not perform some of the blocks, or perform some of the stepsconcurrently. By way of example only, detecting outlier dark signals andcompensating for such outlier dark signals (block 602 or 614) can beexecuted concurrently with reading out dark signals (block 600 or 612,respectively). Alternatively, block 602, block 606, block 614, block618, or block 626 does not have to be performed in other embodiments inaccordance with the invention.

Note that embodiments of the present invention can be used in differentimage sensor architectures other than the architecture illustrated inFIG. 2. By way of example only, an image sensor can switch which columnsof pixels are connected to which column output circuits each time a rowof pixels is read out of the image sensor. In general, this is done tofragment to some extent the columnar or striped nature of column fixedpattern noise. Embodiments of the present invention can determine andapply column offset corrections based solely on column circuits, evenwhen the column output circuits are being switched from one pixel columnto another. Embodiments of the present invention can determine and applycolumn offset corrections based solely on pixel column. Alternatively,embodiments of the present invention can determine and apply columnoffset corrections to each combination of pixel column and column outputcircuit.

Similarly, an image sensor can have column circuits that each serve twoor more columns of pixels, thereby requiring multiple sample and readcycles to read each rows of pixels. Embodiments of the present inventionapply column offset corrections to each column circuit, or each pixelcolumn, or to each combination of pixel column and column outputcircuit. By way of further examples, multiple column output circuits canbe available for each column of pixels. This may be the case if one setof column circuits is being read out while an alternate set of columncircuits is sampling image signals in preparation for being read out.Again, the present invention is useful and is not restricted even thoughthere is not a one-to-one correspondence between pixel columns andcolumn output circuits.

FIGS. 7A-7B depict a flowchart of a method for detecting outlier darksignals and compensating for the outlier dark signals when generatingcolumn offset corrections in an embodiment in accordance with theinvention. Initially, a global offset window is determined for all ofthe columns in the pixel array (block 700). The global offset windowidentifies a range of acceptable dark signals for all of the columns inthe pixel array. The range of acceptable dark signals includes a centerdark signal, a maximum dark signal, and a minimum dark signal in anembodiment in accordance with the invention.

The center, minimum, and maximum dark signals can be based on expecteddark signals in one embodiment in accordance with the invention.Alternatively, the center, minimum, and maximum dark signals can bebased on measured dark signals in another embodiment in accordance withthe invention. And in another embodiment in accordance with theinvention, the center, minimum, and maximum dark signals can be based onpreviously determined center, minimum, and maximum dark signals. Otherembodiments in accordance with the invention can determine the range ofacceptable dark signals for all of the columns in the pixel arraydifferently.

Next, as shown in block 702, dark signals from a given number of darkreference pixels in each column of pixels are read out and the darksignals analyzed to determine whether the dark signals are within theglobal offset window. A determination is then made at block 704 as towhether or not one or more dark signals from each column is outside theglobal offset window. If so, the dark signals outside the global offsetwindow-are outlier dark signals. The outlier dark signals arecompensated for so that all of the dark signals used to determine theinitial column offsets (block 404 FIG. 4) or the column offsets (block604 FIG. 6A) are within the global offset window (block 706).

The outlier dark signals are compensated for by substituting a previousdark signal, a selected dark signal, or estimated dark signal for eachoutlier dark signal in one embodiment in accordance with the invention.In another embodiment in accordance with the invention, the outlier darksignals are compensated for by substituting an average dark signal valuefor each outlier dark signal. In yet another embodiment in accordancewith the invention, a median or mode dark signal is used in place ofeach outlier dark signal. Alternatively, each outlier dark signal can bediscarded and the column offset correction determined with fewer darksignals in another embodiment in accordance with the invention. Otherembodiments in accordance with the invention can compensate for outlierdark signals using different techniques than the ones described herein.

The appropriate column offset corrections are then determined using darksignals that are within the global offset window (block 708). Forexample, the initial column offset corrections (block 404 in FIG. 4) orthe column offset corrections (block 604 in FIG. 6A) are determined atblock 708. And depending on the number of dark signals read out of thedark reference pixels, the updated column offset corrections (block 618in FIG. 6B) may also be determined using signals within the globaloffset window in an embodiment in accordance with the invention.

Next, at block, 710, a determination is made as to whether or not agiven number of dark reference pixels have been read out. The givennumber of dark reference pixels is based on a confidence level in thescaled column offsets or the updated column offsets in one embodiment inaccordance with the invention. Alternatively, the given number of darkreference pixels is based on the factors described with reference toblock 612 in FIG. 6.

If the given number of dark reference pixels has not been reached, theglobal offset window continues to be used to determine outlier darksignals. When the given number of dark reference pixels have been readout, the method continues at block 712 where a local offset window isdetermined for each column in the pixel array. A local offset windowidentifies a range of acceptable dark signals for one column in thepixel array in an embodiment in accordance with the invention. The rangeof acceptable dark signals for a local offset window is based on thedark signals read out of that column in an embodiment in accordance withthe invention. By way of example only, the local offset window can bebased on the average dark signal value for that column.

Dark signals are then read out from a given number of dark referencepixels in each column of pixels and analyzed to determine whether thedark signals are within a respective local offset window (block 714). Adetermination is made at block 716 as to whether or not one or more darksignals from each column is outside the local offset window associatedwith that column. If so, the dark signals outside the local offsetwindows are outlier dark signals. The outlier dark signals arecompensated for so that all of the dark signals used to determine thecolumn offset corrections (block 404 FIG. 4 and block 604 FIG. 6A) orthe updated column offset corrections (block 618 in FIG. 6A) are withintheir respective local offset windows (block 718) in an embodiment inaccordance with the invention. The outlier dark signals can becompensated for by using a technique described in conjunction with block706.

The appropriate column offset corrections are then determined using darksignals within the local offset windows (block 720). In one embodimentin accordance with the invention, the column offset corrections (block604 in FIG. 6A) or the updated column offset corrections (block 618 inFIG. 6B) are determined at block 720. Next, at block, 722, adetermination is made as to whether or not one or more local offsetwindows is to be updated. If not, the method returns to block 714.

When one or more local offset windows are to be updated, one or moreupdated local offset windows are produced, as shown in block 724. Alocal offset window is updated by analyzing the dark signals read out ofa column and comparing them to the range of acceptable dark signalsincluded in the local offset window in an embodiment in accordance withthe invention. The range of acceptable dark signals is adjusted based onthe levels of the dark signals read out of the column. The range ofacceptable dark signals can be increased, decreased, the minimum andmaximum dark signals changed, or combinations of these can be used toupdate a local offset window in one or more embodiments in accordancewith the invention.

The updated local offset window or windows are stored in memory at block726. The method then returns to block 714.

Embodiments in accordance with the invention are not limited to theblocks and the order of the blocks shown in FIGS. 7A-7B. Otherembodiments in accordance with the invention can perform additionalsteps, not perform some of the blocks, or perform some of the stepsconcurrently. By way of example only, block 726 does not have to beperformed in other embodiments in accordance with the invention.

Referring now to FIG. 8, there is shown an exemplary plot of darksignals read out of each column that is used to depict local offsetwindows in an embodiment in accordance with the invention. Multiple darksignals 800 are read out of each column (C₀, C₁, C_(N)) in the pixelarray. Local offset windows 802, 804, 806, 808, 810 are determined forthe columns (C₀, C₁, C_(N)). Each local offset window defines a range ofacceptable dark signals for its respective column. Local offset window802 for column C₀ has a maximum dark signal 812 and a minimum darksignal 814 for that column. Local offset window 804 for column C₁ has amaximum dark signal 816 and a minimum dark signal 818 for that column.Similarly, local offset windows 806, 808, 810 for columns C₂, C_(N-1),and C_(N) have maximum dark signals 820, 822, 824 and minimum darksignals 826, 828, 830, respectively, for those columns.

As shown in FIG. 8, some of the columns have outlier dark signals.Column C₀ has one outlier dark signal 832. Both column C₂ and ColumnC_(N-1) have two outlier dark signals 834, 836 and 838, 840,respectively. Columns C₁ and column C_(N) do not have any outlier darksignals. The outlier dark signals 832, 834, 836, 838, 840 can becompensated for using techniques described earlier in conjunction withFIGS. 7A-7B.

Local offset windows 802, 804, 806, 808, 810 can be adjusted using thetechniques described herein. A local offset window is updated byadjusting the range of acceptable dark signals. For example, the rangecan be increased, decreased, the minimum and maximum dark signalschanged, or combinations of these can be used to update a local offsetwindow in one or more embodiments in accordance with the invention.

The invention has been described in detail with particular reference tocertain preferred embodiments thereof, but it will be understood thatvariations and modifications can be effected within the spirit and scopeof the invention.

Even though specific embodiments of the invention have been describedherein, it should be noted that the application is not limited to theseembodiments. In particular, any features described with respect to oneembodiment may also be used in other embodiments, where compatible. Andthe features of the different embodiments may be exchanged, wherecompatible.

PARTS LIST

-   100 image capture device-   102 light-   104 imaging stage-   106 image sensor-   108 processor-   110 memory-   112 display-   114 other input/output devices-   200 pixel array-   202 photoactive pixels-   204 dark reference pixels-   206 opaque layer or light shield-   208 column decoder-   210 row decoder-   212 digital logic-   214 column output circuits-   216 analog front end circuit-   218 memory-   220 control register-   222 scaling circuitry-   224 timing generator-   300 analog to digital converter-   302 analog signal processor-   304 variable gain amplifier-   306 variable gain amplifier-   800 dark signals-   802 local offset window-   804 local offset window-   806 local offset window-   808 local offset window-   810 local offset window-   812 maximum dark signal-   814 minimum dark signal-   816 maximum dark signal-   818 minimum dark signal-   820 maximum dark signal-   822 maximum dark signal-   824 maximum dark signal-   826 minimum dark signal-   828 minimum dark signal-   830 minimum dark signal-   832 outlier dark signal-   834 outlier dark signal-   836 outlier dark signal-   838 outlier dark signal-   840 outlier dark signal

1. A method for compensating for column fixed pattern noise in an imagesensor, wherein the image sensor includes a plurality of photoactivepixels and a plurality of dark reference pixels arranged in rows andcolumns to form a pixel array, the method comprising: reading out, at afirst gain level of pixel readout circuitry, a dark signal value from agiven number of dark reference pixels in each column; determining aninitial column offset correction for one or more columns in the arrayusing respective dark signals read out at the first gain level;detecting changes to different gain levels of the pixel readoutcircuitry; repeatedly scaling the initial column offset corrections togenerate column offset corrections for each of the different gain levelsin response to each detected change to a respective different gainlevel; and adjusting the column offset corrections for each of thedifferent gain levels based on reading out a varying integer number ofthe dark reference pixels in each column, wherein the varying integernumber varies based on additional data.
 2. The method of claim 1,wherein the initial column offset corrections are scaled based on anamount of change between a respective different gain level and the firstgain level.
 3. The method of claim 1, wherein determining an initialcolumn offset correction for one or more columns in the pixel arrayusing respective dark signals read out at the first gain level comprisescalculating an initial column offset correction for each column in thepixel array by averaging respective dark signals read out at the firstgain level to produce an average dark signal level.
 4. The method ofclaim 1, wherein repeatedly scaling the initial column offsetcorrections in response to each detected change to a respectivedifferent gain level comprises repeatedly scaling the initial columnoffset corrections in response to each detected change to a respectivedifferent gain setting.
 5. The method of claim 4, wherein the columnoffset corrections are scaled by a ratio of a respective different gainsetting to the first gain setting.
 6. The method of claim 1, whereinscaling the initial column offset corrections in response to eachdetected change to a respective different gain level comprises scalingthe initial column offset corrections in response to each detectedchange to a respective different measured gain.
 7. The method of claim6, wherein the column offset corrections are scaled by a ratio of arespective different measured gain to a first measured gain.
 8. Themethod of claim 1, wherein the additional data includes a noise level ofa first subset of the dark reference pixels in each column that are readout to generate additional dark signal values.
 9. The method of claim 1,wherein the additional data includes whether the pixel array hasinitiated an image capture at a detected gain level since the pixelarray was last powered off, wherein the detected gain level is among thedifferent gain levels.
 10. A method compensating for column fixedpattern noise in an image sensor, wherein the image sensor includes aplurality of photoactive pixels and a plurality of dark reference pixelsarranged in rows and columns to form a pixel array, the methodcomprising: reading out, at a first gain level of pixel readoutcircuitry, a dark signal value from a given number of dark referencepixels in each column; determining an initial column offset correctionfor one or more columns in the array using respective dark signals readout at the first gain level; detecting changes to different gain levelsof the pixel readout circuitry; repeatedly scaling the initial columnoffset corrections to generate column offset corrections for each of thedifferent gain levels in response to each detected change to arespective different gain level, wherein the initial column offsetcorrections are scaled based on an amount of change between therespective different gain level and the first gain level; and adjustingthe column offset corrections for each of the different gain levelsbased on reading out a varying integer number of the dark referencepixels in each column, wherein the varying integer number varies basedon additional data.
 11. The method of claim 10, wherein determining aninitial column offset correction for one or more columns in the pixelarray using respective dark signals read out at the first gain levelcomprises calculating an initial column offset correction for eachcolumn in the pixel array by averaging respective dark signals read outat the first gain level to produce an average dark signal level.
 12. Themethod of claim 10, wherein repeatedly scaling the initial column offsetcorrections in response to each detected change to a respectivedifferent gain level comprises repeatedly scaling the initial columnoffset corrections in response to each detected change to a respectivedifferent gain setting.
 13. The method of claim 12, wherein the columnoffset corrections are scaled by a ratio of a respective different gainsetting to the first gain setting.
 14. The method of claim 10, whereinrepeatedly scaling the initial column offset corrections in response toeach detected change to a respective different gain level comprisesrepeatedly scaling the initial column offset corrections in response toeach detected change to a respective different measured gain.
 15. Themethod of claim 14, wherein the column offset corrections are scaled bya ratio of a respective different measured gain to a first measuredgain.
 16. The method of claim 10, wherein the additional data includes anoise level of a first subset of the dark reference pixels in eachcolumn that are read out to generate additional dark signal values. 17.The method of claim 10, wherein the additional data includes whether thepixel array has initiated an image capture at a detected gain levelsince the pixel array was last powered off, wherein the detected gainlevel is among the different gain levels.